Papers with Task-oriented dialog systems
Benchmarking the Covariate Shift Robustness of Open-world Intent Classification Approaches (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing benchmarks for open-world intent classification focus on the second aspect of the problem. |
| Approach: | They propose two datasets that include utterances useful for evaluating the robustness of open-world models to covariate shift. |
| Outcome: | The proposed datasets provide a more realistic evaluation scenario for open-world intent classification in task-oriented dialog systems. |
XL-NBT: A Cross-lingual Neural Belief Tracking Framework (D18-1)
Copied to clipboard
| Challenge: | a multi-lingual approach to training dialog systems is expensive and tedious, but it can be useful for cross-lingual support. |
| Approach: | They propose to annotate data for multiple languages and train a multi-lingual dialog system for each language. |
| Outcome: | The proposed framework bypasses the expensive human annotation and achieves promising results. |
An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction (D19-1)
Copied to clipboard
Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, Jason Mars
| Challenge: | Task-oriented dialog systems need to know when a query falls outside their range of supported intents. |
| Approach: | They propose a dataset that includes queries that are out-of-scope and 150 intent classes over 10 domains. |
| Outcome: | The proposed dataset includes queries that are out-of-scope, i.e., queries that do not fall into any of the system’s supported intents. |
Robust Zero-Shot Cross-Domain Slot Filling with Example Values (P19-1)
Copied to clipboard
| Challenge: | Task-oriented dialog systems rely on deep learning-based slot filling models . little to no training data for target domains may be available or schemas may not be aligned . |
| Approach: | They propose to use slot descriptions and examples of slot values to learn slot semantic representations that are transferable across domains and robust to misaligned schemas. |
| Outcome: | The proposed model outperforms state-of-the-art models on two multi-domain datasets on low-data setting. |